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DETAILED ACTION 

Drawings 

1 . The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference character(s) not mentioned in the 
description: reference number "135" in Fig. 1A, reference number "500" in Fig. 5. 

2. The drawings are objected to because the process in Fig. 8 is referenced 
"730" instead of - 800 - as described by the specifications in page 25, line 18. 
Also the process in Fig. 9 is referenced as "840" instead of - 900 - as described 
by the specifications in page 26, line 19. 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are 
required in reply to the Office action to avoid abandonment of the application. 
Any amended replacement drawing sheet should include all of the figures 
appearing on the immediate prior version of the sheet, even if only one figure is 
being amended. The figure or figure number of an amended drawing should not 
be labeled as "amended." If a drawing figure is to be canceled, the appropriate 
figure must be removed from the replacement sheet, and where necessary, the 
remaining figures must be renumbered and appropriate changes made to the 
brief description of the several views of the drawings for consistency. Additional 
replacement sheets may be necessary to show the renumbering of the remaining 
figures. Each drawing sheet submitted after the filing date of an application must 
be labeled in the top margin as either "Replacement Sheet" or "New Sheet" 
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pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, 
the applicant will be notified and informed of any required corrective action in the 
next Office action. The objection to the drawings will not be held in abeyance. 

Specification 

3. The disclosure is objected to because of the following informalities: 
Page 6, line 1: "BEST MODE FOR CARRYING OUT THE INVENTION" 
should be - DETAILED DESCRIPTION OF THE INVENTION --. 
Page 17, line 15: "180may" should be - 180 may -. 

Page 17, line 17: "processing may be include" should be - processing 
may include — . 

Page 25, lines 14 and 16: "process 730" should be - process 800 -. 
Page 26, lines 15 and 17: "process 840" should be - process 900 -. 

Appropriate correction is required. 

Claim Objections 

4. Claims 5 and 31 are objected to because of the following informalities: 
Re Claim 5, line 3: "cloud of said subset" should be -- cloud of a subset -. 
Re Claim 31 , line 6: A period should be inserted after "at said classifier". 



Appropriate correction is required. 
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Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 

U.S.C. 102 that form the basis for the rejections under this section made in this 

Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before 
the invention by the applicant for patent, except that an international application filed under 
the treaty defined in section 351 (a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

6. Claims 1, 2, 4, 6, 9, 11, 13-15, 18, 23, 24, 26, and 28-31 are rejected 
under 35 U.S.C. 102(e) as being anticipated by Carrot et al (US 6909792 B1 ). 
Re Claim 1 : Carrot discloses a method for visual-based recognition (see 
Abstract, lines 1, and 11-15) of an object / breast, said method comprising 
receiving depth data (see Fig. 7, col. 6, line 21 , depth or z') for at least a pixel of 
an image of an object, said depth data comprising information relating to a 
distance from a visual sensor (see col. 6, lines 5-26, distance from the ultrasonic 
scanner to the different parts of the breast tissue) to a portion of said object / 
breast shown at said pixel; generating a plan-view image / slice (167) (see Fig. 7, 
col. 9, lines 22-23) based in part on said depth data; extracting a plan-view 
template / entire slice (167) (see Fig. 7, Abstract, lines 11-15, historical images, 
registered images from previous scanning of the breast tissue, the template may 
be the entire plan-view image itself) from said plan-view image; and processing 
said plan-view template at a classifier / correlator (30) (see Fig. 1 , col. 2 line 45- 
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47), wherein said classifier is trained to make a decision (see Abstract, lines 11- 
15, tumor growth or shrinkage) according to pre-configured parameters. 

Re Claim 2: Carrot discloses receiving non-depth data / color (see col. 2, line 57, 
multicolor imagery) for said pixel. 

Re Claim 4: Carrot discloses selecting a subset / ROI of said depth data / 
multicolored data based on foreground segmentation / thresholding multicolored 
data (see col. 3, lines 58-60, getting ROI with thresholding gives features of 
breast, col. 3, lines 40-43). 

Re Claim 6: Carrot discloses receiving non-depth data / multicolored data for said 
pixel, and wherein said foreground segmentation / thresholding multicolored data 
is based at least in part on said non-depth data (see col. 1 , lines 64-66, Abstract, 
lines 13-15, thresholding on ROI colored image, col. 3, lines 58-60). 

Re Claim 9: Carrot discloses extracting said plan-view template from said plan- 
view image is based at least in part on object tracking / ROI (see col. 3, lines 25- 
36 and 40-43, tracking ROI). 

Re Claims 1 1 : Carrot discloses said plan-view image is based in part on said 
non-depth data (see col. 2, line 57, multicolor imagery, each slice or plan-view 
will also have multicolor imagery). 
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Re Claim 13: Carrot discloses said plan-view image comprises a value based at 
least in part on an estimate of height / depth z' of a portion of said object / breast 
above a surface /pressure plate (83, 84) (see Fig. 3, col. 5, lines 59-61, the depth 
is considered as an estimate of height a result of the breast being on a pressure 
plate). 

Re Claim 14: Carrot discloses said plan-view image comprises a value based at 
least in part on color data for a portion of said object (see Abstract, lines 13-15, 
col. 1, lines 64-65, col. 3, lines 4-8, col. 2, line 57, multicolor imagery, each slice 
or plan-view will also have multicolor imagery). 

Re Claim 15: Carrot discloses said plan-view image comprises a value based at 
least in part on a count of pixels / ROI obtained by said visual sensor and 
associated with said object (see col. 3, lines 27-44, col. 4, lines 26-28, ROI has a 
certain amount or count of pixels). 

Re Claim 18: Carrot discloses performing height normalization / depth z' based 
on pressure plate (83, 84) on said plan-view template / slice (167) (see Pigs. 3 
and 7, col. 5, lines 59-61 , the depth is considered as a height normalization as a 
result of the breast being on a pressure plate, the three dimensional image is 
therefore dependent upon the height normalization and therefore each template 
or slice is dependent upon the height normalization). 
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Re Claim 23: Carrot discloses a visual-based recognition system comprising a 
visual sensor (20) (see Fig. 1, col. 2, lines 41-45, Abstract, lines 1, and 11 -15) for 
capturing depth data (see Fig. 7, col. 6, line 21 , depth or z') for at least a pixel of 
an image of an object / breast, said depth data comprising information relating to 
a distance from said visual sensor (see col. 6, lines 5-26, distance from the 
ultrasonic scanner to the different parts of the breast tissue) to a portion of said 
object visible at said pixel; a plan-view image generator (20,24) (see col. 6, lines 
27-55) for generating a plan-view image / slice (167) (see Fig. 7, col. 9, lines 22- 
23) based on said depth data; a plan-view template generator (20,24) (see col. 6, 
lines 45-50, access historical images) for generating a plan-view template / slice 
(167) (see Fig. 7, Abstract, lines 11-15, historical images, registered images from 
previous scanning of the breast tissue, the template may be the entire plan-view 
image itself) based on said plan-view image; and a classifier / correlator (30) (see 
Fig. 1 , col. 2 line 45-47) for making a decision concerning recognition (see 
Abstract, . lines 11-15, tumor growth or shrinkage) of said object, wherein said 
classifier is trained to make a decision according to pre-configured parameters. 

Re Claim 24: Carrot discloses said visual sensor is also for capturing non-depth 
data / color (see col. 2, line 57, multicolor imagery). 

Re Claim 26: Carrot discloses a pixel subset selector (52, 24) for selecting a 
subset / ROI of pixels of said image, wherein said pixel subset selector is 
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operable to select said subset of pixels based on foreground segmentation / 
thresholding multicolored data (see Figs. 1 and 2a, see col. 3, lines 58-60, 
getting ROI with thresholding gives features of breast, col. 3, lines 40-43). 

Re Claim 28: Carrot discloses said plan-view image is based in part on said non- 
depth data (see col. 2, line 57, multicolor imagery, each slice or plan-view will 
also have multicolor imagery). 

Re Claim 29: Carrot discloses to generate a three-dimensional. point cloud / three 
dimensional data set based on said depth data / direction z', wherein a point of 
said three-dimensional point cloud comprises a three-dimensional coordinate / x', 
y', z' (see Fig. 7, col. 3, lines 4-7, col. 6, lines 20-21). 

Re Claim 30: Carrot discloses to divide said three-dimensional point cloud / three 
dimensional data set into a plurality of slices such that a plan-view image (167) 
may be generated for at least one slice of said plurality of slices (see Fig. 7, col. 
3, lines 4-7, col. 9, lines 16-27). 

Re Claim 31 : Carrot discloses to extract a plan-view template / slice (167) from at 
least two plan-view images / plurality of slices corresponding to different slices of 
said plurality of slices, wherein said plan-view template comprises a 
transformation / summing data points of at least a portion of said plan-view 
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images / entire slice, such that said plan-view template is processed at said 
classifier (see Fig. 7, col. 9, lines 22-27). 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

8. Claims 5, 7, 8, 32, 33, 35-37, 39 and 40 are rejected under 35 

U.S.C. 103(a) as being unpatentable over Carrot as applied to claims 1 and 23 
above. 

Re Claim 5 and 32 respectively: Carrot, as recited in claim 5 f discloses 
generating a three-dimensional point cloud / three dimensional data set of said 
subset / ROI of pixels based on said depth data / direction z' (col. 3, lines 27-37 
and 40-43, col. 6, lines 20-21 , wherein a point of said three-dimensional point 
cloud comprises a three-dimensional coordinate / x\ y\ z' (see Fig. 7, col. 3, lines 
4-7); partitioning said three-dimensional point cloud into a plurality of vertically 
oriented bins; and mapping at least a portion of points of said plurality of 
vertically oriented bins into at least one said plan-view image / ROI or slice 
addition based on said three- dimensional coordinates, wherein said plan-view 
image is a two-dimensional representation / slice of said three-dimensional point 
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cloud comprising at least one pixel corresponding to at least one vertically 
oriented bin of said plurality of vertically oriented bins (see col. 3, lines 27-37 and 
40-43, ROI may cover these vertical bins, col. 9, lines 22-26, addition of all the 
possible slices will then cover all the bins). 

Although the partitioning step of Carrot's three dimensional cloud is not 
specifically disclosed, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to have such a feature where the vertical 
bin is each depth or z' axis point, so for example, if the point [x',y'] is looked at, all 
the same points [x',y'] along the different z' values creates a vertical bin. 

As to claim 32, all the limitations are taught by Carrot in the same manner 
as Carrot taught claims 1 , 5, and 6 respectively above. 

Re Claim 7: Carrot discloses dividing said three-dimensional point cloud into a 
plurality of slices (164, 167), and wherein said generating said plan-view image / 
slice (167) is performed for at least one slice of said plurality of slices (see Fig. 7, 
col. 3, lines 4-7). 

Re Claim 8: Carrot discloses extracting a plan view template / slice (167) from at 
least two plan-view images / plurality of slices corresponding to different slices of 
said plurality of slices, wherein said plan-view template comprises a 
transformation / summing data points of at least a portion of said plan-view 
images / entire slice, such that said plan-view template is processed at said 
classifier (see Fig. 7, col. 9, lines 22-27). 
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Claims 7 and 8 are dependent on claim 5. 

Re Claim 33: Carrot discloses said three-dimensional point cloud / three 
dimensional data set and said plan-view image / entire slice (167) are also based 
at least in part on non-depth data / multicolored data (see Fig. 7, col. 1 , lines 64- 
65, col. 3, lines 4-7, Abstract, lines 11-15). 

Re Claim 35: Carrot discloses extracting a plan-view template / entire slice (167) 
from said plan-view image / slice (167), wherein said plan view template 
comprises a transformation / summing data points of at least a portion of said 
plan view image / entire slice (1 67), and such that said plan-view template is 
processed at said classifier (see Fig. 7, col. 9, lines 22-27). 

Re Claim 36: Carrot discloses dividing said three-dimensional point cloud / three 
dimensional data set of into a plurality of slices, and wherein said mapping / 
summing data points at least a portion of points comprises mapping points within 
a slice of said plurality of slices of said three-dimensional point cloud into said 
plan-view image / slice (see col. 3, lines 4-7, col. 9, lines 22-27). 

Re Claim 37: Carrot discloses comprising extracting a plan-view template / entire 
slice (167) from said plan-view image / slice, wherein said plan view template 
comprises a transformation / summing data points of at least a portion of said 
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plan view image / entire slice (167), such that said plan-view template is 
processed at said classifier (see Fig. 7, col. 9, lines 22-27). 

Re Claim 39: Carrot discloses said plan-view image / slice is generated from a 
subset / ROI of pixels of said image selected based on foreground segmentation 
/ thresholding multicolored data (see col. 3, lines 58-60, getting ROI with 
thresholding gives features of breast, col. 3, lines 40-43). 

Re Claim 40: Carrot discloses extracting a plan view template / entire slice (167) 
from at least two plan view images corresponding to different slices of said 
plurality of slices, wherein said plan view template comprises a transformation / 
summing of data points of at least a portion of said plan view images / entire slice 
(167), such that said plan-view template is processed at said classifier (see Fig. 
7, col. 9, lines 22-27). 

Claims 33, 35-37, 39 and 40 are dependent on claim 32. 

9. Claims 3, 10, 12, 16, 17, 19-22, 25, 27, 34, and 38 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Carrot as applied to claims 1, 23 and 
32 above, and further in view of Li et al (US 2003/0108244 A1). 

However, Carrot fails to teach of fairly suggest depth data is determined 
by stereopsis, that the classifier is a support vector machine, that the plan-view 
template is a vector basis obtained by principal component analysis (PCA), and 
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that a decision is to distinguish between a human, non-human, and different 
bodies. 

Li, as recited in claim 3 and claim 34 respectively , discloses said depth 
data using stereopsis / one or more cameras based on image correspondences 
(see page 6, right col., line 7). 

Li, as recited in claim 10 t discloses said classifier is a support vector 
machine / SVM's (see page 1, paragraph [0008], lines 18-24, slices or plan-view 
image will be affected by SVM's). 

Li, as recited in claim 12 , discloses said object is a person (see page 1 , 
paragraph [0005], lines 17-19, paragraph [0007], last two lines, "face and non- 
face classification"). 

Li. as recited in claim 16 , discloses Re Claim 16: The method as recited in 
Claim 1 wherein said plan-view template is represented in terms of a vector basis 
/ SVM's (see page 1, paragraph [0008], lines 18-24, since slices or plan-view 
image will be affected by SVM's, so will the templates since the template is the 
entire slice). 

Li, as recited in claim 17 , discloses said vector basis is obtained through 
principal component analysis (PCA) (see page 1, paragraph [0008], lines 18-24, 
"PCA as they rotate and use the SVM's"). 

Li, as recited in claim 19 , discloses said decision is to distinguish between 
a human and a non-human (see page 1, paragraph [0005], lines 17-19, 
paragraph [0007], last two lines, "face and non-face classification"). 
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Li, as recited in claim 20 . discloses said decision is to distinguish between 
a plurality of different human body orientations (see page 2, paragraph [0016], 
page 4, paragraph [0039], lines 13-15, classes of view ranges being left profile, 
left half profile, frontal, etc.). 

Li, as recited in claim 21 . discloses said decision is to distinguish between 
a plurality of different human body poses / face poses (see page 7, paragraph 
[0073], last two lines, page 4, paragraph [0039], lines 13-15). 

Li, as recited in claim 22 . discloses said decision is to distinguish between 
a plurality of different classes / position of people (see page 2, paragraph [0016], 
page 4, paragraph [0039], lines 13-15, "classes of view ranges"). 

Li, as recited in claim 25 . discloses said visual sensor determines said 
depth data using stereopsis / one or more cameras based on image 
correspondences (see page 6, right col., line 7). 

Li, as recited in claim 27 and claim 38 respectively , discloses said 
classifier is a support vector machine / SVM's (see page 1 , paragraph [0008], 
lines 18-24, "PCA as they rotate and use the SVM's for multi-pose face 
detection"). 

Therefore, in view of Li, it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to modify Carrot's visual-based 
recognition method and system by including the capabilities of stereopsis for 
determining depth data, the classifier being a support vector machine, the plan- 
view template being a vector basis obtained by principal component analysis 
(PCA), and a decision by the classifier being to distinguish between a human, 
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non-human, and different bodies, to the receiving depth data and processing the 
plan-view template steps of Carrot in order to detect a person's face in real time, 
in input images containing either frontal or non-frontal views regardless of the 
scale or illumination conditions associated with the face. 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Geng et al discloses a three-dimensional ear biometrics 
system and method; Wilson et al discloses an architecture for controlling a 
computer using hand gestures; Bradski et al discloses a method, apparatus and 
system for using computer vision to identify facial characteristics; Krebs et al 
discloses methods and systems for distinguishing individuals utilizing anatomy 
and gait parameters; Cohen et al discloses gesture-controlled interfaces for self- 
service machines and other applications; Park et al discloses apparatus and 
method for extracting object based on feature matching between segmented 
regions in images; Kunieda discloses a radiotherapy device; Freeman et al 
discloses a system for reconstructing the 3-dimensional motions of a human 
figure from a monocularly-viewed image sequence; Krumm et al discloses a 
system and process for identifying and locating people or objects in a scene by 
selectively clustering three-dimensional regions; Harville et al discloses a plan- 
view projections of depth data for object tracking; Mihara et al discloses an image 
recognition method and apparatus. 
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1 1 . Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Bernard Krasnic whose telephone number is 
(571) 270-1357. The examiner can normally be reached on Mon-Thur 7:30am- 
5:00pm and every other Friday 7:30am-4:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Jong-Suk (James) Lee can be reached on (571) 272- 
7044. The fax phone number for the organization where this application or 
proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 

Bernard Krasnic 
December 7, 2006 
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